Computer aided diagnosis of diabetic retinopathy based on multi-view joint learning.

Diabetic retinopathy (DR) is a kind of ocular complication of diabetes, and its degree grade is an essential basis for early diagnosis of patients. Manual diagnosis is a long and expensive process with a specific risk of misdiagnosis. Computer-aided diagnosis can provide more accurate and practical treatment recommendations. In this paper, we propose a multi-view joint learning DR diagnostic model called RT2Net, which integrates the global features of fundus images and the local detailed features of vascular images to reduce the limitations of single fundus image learning. Firstly, the original image is preprocessed using operations such as contrast-limited adaptive histogram equalization, and the vascular structure of the extracted DR image is segmented. Then, the vascular image and fundus image are input into two branch networks of RT2Net for feature extraction, respectively, and the feature fusion module adaptively fuses the feature vectors' output from the branch networks. Finally, the optimized classification model is used to identify the five categories of DR. This paper conducts extensive experiments on the public datasets EyePACS and APTOS 2019 to demonstrate the method's effectiveness. The accuracy of RT2Net on the two datasets reaches 88.2% and 85.4%, and the area under the receiver operating characteristic curve (AUC) is 0.98 and 0.96, respectively. The excellent classification ability of RT2Net for DR can significantly help patients detect and treat lesions early and provide doctors with a more reliable diagnosis basis, which has significant clinical value for diagnosing DR.

Research on Multimodal Fusion of Temporal Electronic Medical Records.

The surge in deep learning-driven EMR research has centered on harnessing diverse data forms. Yet, the amalgamation of diverse modalities within time series data remains an underexplored realm. This study probes a multimodal fusion approach, merging temporal and non-temporal clinical notes along with tabular data. We leveraged data from 1271 myocardial infarction and 6450 stroke inpatients at a Beijing tertiary hospital. Our dataset encompassed static, and time series note data, coupled with static and time series table data. The temporal data underwent a preprocessing phase, padding to a 30-day interval, and segmenting into 3-day sub-sequences. These were fed into a long short-term memory (LSTM) network for sub-sequence representation. Multimodal attention gates were implemented for both static and temporal subsequence representations, culminating in fused representations. An attention-backtracking module was introduced for the latter, adept at capturing enduring dependencies in temporal fused representations. The concatenated results were channeled into an LSTM to yield the ultimate fused representation. Initially, two note modalities were designated as primary modes, and subsequently, the proposed fusion model was compared with comparative models including recent models such as Crossformer. The proposed model consistently exhibited superior predictive prowess in both tasks. Removing the attention-backtracking module led to performance decline. The proposed model consistently shows excellent predictive capabilities in both tasks. The proposed method not only effectively integrates data from the four modalities, but also has a good understanding of how to handle irregular time series data and lengthy clinical texts. An effective method is provided, which is expected to be more widely used in multimodal medical data representation.

Representation of time-varying and time-invariant EMR data and its application in modeling outcome prediction for heart failure patients.

OBJECTIVE: To represent a patient record with both time-invariant and time-varying features as a single vector using an end-to-end deep learning model, and further to predict the kidney failure (KF) status and mortality of heart failure (HF) patients. MATERIALS AND METHODS: The time-invariant EMR data included demographic information and comorbidities, and the time-varying EMR data were lab tests. We used a Transformer encoder module to represent the time-invariant data, and refined a long short-term memory (LSTM) with a Transformer encoder attached to the top to represent the time-varying data, taking the original measured values and their corresponding embedding vectors, masking vectors, and two types of time intervals as inputs. The proposed representations of patients with time-invariant and time-varying data were used to predict KF status (949 out of 5268 HF patients diagnosed with KF) and mortality (463 in-hospital deaths) for HF patients. Comparative experiments were conducted between the proposed model and some representative machine learning models. Ablation experiments were also performed around the time-varying data representation, including replacing the refined LSTM with the standard LSTM, GRU-D and T-LSTM, respectively, and removing the Transformer encoder and the time-varying data representation module, respectively. The visualization of the attention weights of the time-invariant and time-varying features was used to clinically interpret the predictive performance. We used the area under the receiver operating characteristic curve (AUROC), the area under the precision-recall curve (AUPRC), and the F1-score to evaluate the predictive performance of the models. RESULTS: The proposed model achieved superior performance, with average AUROCs, AUPRCs and F1-scores of 0.960, 0.610 and 0.759 for KF prediction and 0.937, 0.353 and 0.537 for mortality prediction, respectively. Predictive performance improved with the addition of time-varying data from longer time periods. The proposed model outperformed the comparison and ablation references in both prediction tasks. CONCLUSIONS: Both time-invariant and time-varying EMR data of patients could be efficiently represented by the proposed unified deep learning model, which shows higher performance in clinical prediction tasks. The way to use time-varying data in the current study is hopeful to be used in other kinds of time-varying data and other clinical tasks.

Spatially Resolved Stimulation for the Controlled Debromination in Single Molecules on a Surface.

A controlled chemical reaction on a specific bond in a single molecule is an inevitable step toward atomic engineering and fabrication. Here, we explored the debromination of a single 9,10-dibromoanthracene (DBA) molecule on a surface as stimulated by the voltage pulse through the tip of a scanning tunneling microscope (STM). A voltage threshold of about 2.2 V is obtained, and the nature of single-electron process is revealed. The spatially resolved debromination yield is obtained as a function of the pulse magnitude, which presents strong asymmetry for the two C-Br bonds. The optimal stimulation parameters including the pulse magnitude and the tip locations are suggested. The distinct dynamics in dissociation of the two bonds are illustrated by their energy diagrams and recoil paths, as derived by the first-principles density functional theory (DFT) calculation. The influence of the local electric field due to the STM tip on the dissociation of the C-Br bond has also been discussed. The study presents detailed practice for the controlled debromination in a single DBA molecule, which may lead to automated atomic engineering and fabrication of artificial nanostructures in the future.

Sequential Data-Based Patient Similarity Framework for Patient Outcome Prediction: Algorithm Development.

BACKGROUND: Sequential information in electronic medical records is valuable and helpful for patient outcome prediction but is rarely used for patient similarity measurement because of its unevenness, irregularity, and heterogeneity. OBJECTIVE: We aimed to develop a patient similarity framework for patient outcome prediction that makes use of sequential and cross-sectional information in electronic medical record systems. METHODS: Sequence similarity was calculated from timestamped event sequences using edit distance, and trend similarity was calculated from time series using dynamic time warping and Haar decomposition. We also extracted cross-sectional information, namely, demographic, laboratory test, and radiological report data, for additional similarity calculations. We validated the effectiveness of the framework by constructing k-nearest neighbors classifiers to predict mortality and readmission for acute myocardial infarction patients, using data from (1) a public data set and (2) a private data set, at 3 time points-at admission, on Day 7, and at discharge-to provide early warning patient outcomes. We also constructed state-of-the-art Euclidean-distance k-nearest neighbor, logistic regression, random forest, long short-term memory network, and recurrent neural network models, which were used for comparison. RESULTS: With all available information during a hospitalization episode, predictive models using the similarity model outperformed baseline models based on both public and private data sets. For mortality predictions, all models except for the logistic regression model showed improved performances over time. There were no such increasing trends in predictive performances for readmission predictions. The random forest and logistic regression models performed best for mortality and readmission predictions, respectively, when using information from the first week after admission. CONCLUSIONS: For patient outcome predictions, the patient similarity framework facilitated sequential similarity calculations for uneven electronic medical record data and helped improve predictive performance.

Exploring the Relationship Between Anxiety, Depression, and Sleep Disturbance Among HIV Patients in China From a Network Perspective.

Background: Mental disorder of people living with HIV (PLWH) has become a common and increasing worldwide public health concern. We aimed to explore the relationship between anxiety, depression, and sleep disturbance for PLWH from a network perspective. Methods: The network model featured 28 symptoms on the Hospital Anxiety and Depression scale questionnaire and Pittsburgh Sleep Quality Index questionnaire in a sample of 4,091 HIV-infected persons. Node predictability and strength were computed to assess the importance of items. We estimated and compared 20 different networks based on subpopulations such as males and females to analyze similarities and differences in network structure, connections, and symptoms. Results: Several consistent patterns and interesting differences emerged across subgroups. Pertaining to the connections, some symptoms such as S12-S13 ("sleepy"-"without enthusiasm") shown a strong positive relationship, indicating that feeling sleepy was a good predictor of lacking enthusiasm, and vice versa. While other symptoms, such as A3-D3 ("worried"-"cheerful"), were negatively related in all networks, revealing that nodes A3 and D3 were bridge symptoms between anxiety and depression. Across all subgroups, the most central symptom was A7 "panic" and S2 "awake", which had the greatest potential to affect an individual's mental state. While S3 "bathroom" and S5 "cough or snore" shown consistent lower node importance, which would be of limited therapeutic use. Conclusions: Mental conditions of PLWH varied considerably among subgroups, inspiring psychiatrists and clinicians that personalized invention to a particular subgroup was essential and might be more effective during treatment than adopting the same therapeutic schedule.